Model-based decision support systems (DSS) improve performance in many contexts that are data-rich, uncertain, and require repetitive decisions. But such DSS are often not designed to help users understand and internalize the underlying factors driving DSS recommendations. Users then feel uncertain about DSS recommendations, leading them to possibly avoid using the system. We argue that a DSS must be designed to induce an alignment of a decision maker's mental model with the decision model embedded in the DSS. Such an alignment requires effort from the decision maker and guidance from the DSS. We experimentally evaluate two DSS design characteristics that facilitate such alignment: (i) feedback on the upside potential for performance improvement and (ii) feedback on corrective actions to improve decisions. We show that, in tandem, these two types of DSS feedback induce decision makers to align their mental models with the decision model, a process we call deep learning, whereas individually these two types of feedback have little effect on deep learning. We also show that deep learning, in turn, improves user evaluations of the DSS. We discuss how our findings could lead to DSS design improvements and better returns on DSS investments.
We study the process by which model-based decision support systems (DSSs) influence managerial decision making in the context of marketing budgeting and resource allocation. We focus on identifying whether and how DSSs influence the decision process (e.g., cognitive effort deployed, discussion quality, and decision alternatives considered) and, as a result, how these DSSs influence decision outcomes (e.g., profit and satisfaction both with the decision process and the outcome). We study two specific marketing resource allocation decisions in a laboratory context: sales effort allocation and customer targeting. We find that decision makers who use high-quality, model-based DSSs make objectively better decisions than do decision makers who only have access to a generic decision tool (Microsoft Excel). However, their subjective evaluations (perceptions) of both their decisions and the processes that lead to those decisions do not necessarily improve as a result of DSS use. And expert judges, serving as surrogates for top management, have a difficult time assessing the objective quality of those decisions. Our results suggest that what managers get from a high-quality DSS may be substantially better than what they see. To increase the inclination for managerial adoption and use of DSS, we must get users to "see" the benefits of using a DSS. Our results also suggest two ways to bridge the perception-reality gap: (1) improve the perceived value of the decision process by designing DSSs both to encourage discussion (e.g., by providing explanation and support for alternative recommendations) as well as to reduce the perceived complexity of the problem so that managers invest more cognitive effort in exploring additional options and (2) provide feedback on the likely market/business outcomes of various decision options.
Creativity support systems (CSS) aim at enhancing the creativity of users. There is an emerging stream of research in which the effects of CSS on the creative output of respondents are measured. In this research, it is important to make a clear distinction between the dependent variable, creative output, and the independent variable use of CSS. Furthermore, the research design should take the potential effect of other factors on creative output into account, most notably, creative ability as a trait of the respondents. An experimental study on the value of creativity support systems was recently reported in MIS Quarterly (Massetti 1996). That study yielded interesting insights with respect to the value of CSS. However, because of the methodology applied in analyzing the data, the study underestimated the effects of CSS on the creative output of decision makers. In this note, Massetti's experiment is positioned in the broader perspective of current research in the area of CSS, and an alternative framework for analyzing the data is proposed.